WE CLAIM: 

1. A graphical user interface comprising: 

a rendered image of at least one graphical object 
wherein the graphical object using a number of pixels on 
5 a display device; 

a color value stored for each pixel in the display 
device; and 

object identification data stored with each pixel 
covered by the rendered image wherein the object 
10 identification data uniquely identifies the graphical 
object located at the pixel. 

2. The graphical user interface of claim 1 wherein 
the rendered image comprises a three-dimensional image 
comprising at least two graphical objects that are co- 
located at at least one point in an x-y view plane, 

5 wherein only one of the co-located graphical objects is 
visible at a pixel corresponding to the at least one 
point and the object identification data for the pixel 
identifies the visible graphical object. 

3. The graphical user interface of claim 1 further 
comprising processes for writing the object 
identification data with the color value to a frame 
buffer when drawing the rendered image of the at least 

5 one graphical object on the display device. 

4 . The graphical user interface of claim 1 further 
comprising processes for reading the object 
identification data with the color value from the frame 
buffer. 

5. The graphical user interface of claim 1 further 
comprising a cursor pointing to a particular coordinate 
of the graphical display device, movement of which is 
controlled by user input. 

-21- 

\\\BO - 8S528/3 - #25346 vl 



6. The graphical user interface of claim 5 further 
comprising processes for reading the object 
identification data with the color value from the frame 
buffer of a pixel associated with the particular 
coordinate pointed to by the cursor. 

7 . The graphical user interface of claim 6 further 
comprising an interface for exporting the object 
identification information to an external process. 

8. The graphical user interface of claim 6 further 
comprising a metadata information display object 
displayed on the display in response to the object 
identification data of the pixel associated with the 
particular coordinate pointed to by the cursor. 

9. The graphical user interface of claim 6 further 
comprising an object-identification value indexed data 
structure holding metadata about the an object uniguely 
identified by the object identification value. 

10. The graphical user interface of claim 6 further 
comprising a metadata information display object 
displayed on the display in response to the object 
identification data of the pixel associated with the 
particular coordinate pointed to by the cursor. 

11. A method for providing information to a program 
using a graphical user interface, the method comprising: 

rendering an image of a plurality of graphical 
objects at specified locations of a two-dimensional 
display device; 

storing a color value for each location in the two- 
dimensional display device; and 

storing object identification data for each of the 
specified locations, wherein the object identification 
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data uniquely identifies one of the graphical objects at 
the at least one location. 

12. The method of claim 11 wherein from a 
particular viewpoint, at least two of the graphical 
objects intersect at at least one intersection location 
of image, and the object identification data stored at 
the pixel associated with the intersection location 
identifies an object that is visually closest to the 
viewpoint . 

13. The method of claim 11 further comprising: 
using the object identification data to identify one 

of the intersecting graphical objects. 

14. The method of claim 11 further comprising 
writing the object identification data with the color 
value to a frame buffer when drawing the rendered image 
of the plurality of graphical objects on the two- 
dimensional display device. 

15. The method of claim 11 further comprising 
reading the object identification data with the color 
value from the frame buffer. 

16. The method of claim 11 further comprising: 
receiving user input indicating movement on the two- 
dimensional display device; and 

pointing to a particular location of the two- 
dimensional graphical display device indicated by the 
user input. 

17. The method of claim 16 further comprising 
reading, from the frame buffer, the object identification 
data with the color value of a pixel associated with the 
particular location pointed to. 
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18. The method of claim 17 further comprising 
exporting the object identification information to an 
external process. 

19. The method of claim 17 further comprising 
displaying a metadata information display object on the 
two-dimensional display device in response to the object 
identification data read from the particular location 

5 pointed to. 

20. The method of claim 17 further comprising 
storing metadata about the graphical objects in an 
object-identification value indexed data structure; and 

using the object-identification value read from the 
5 particular location pointed at to index into the data 
structure so as to retrieve metadata about a particular 
object . 

21. The method of claim 20 further comprising 
displaying a metadata information display object 
displayed on the display in response to the object 
identification data. 

22. A computer-readable medium containing 
instructions for controlling a data processing system to 
perform a method of providing information to a program 
object, the method comprising: 

5 rendering an image of a graphical object at 

specified locations of a two-dimensional display device; 

storing a color value for each location in the two- 
dimensional display device; and 

storing object identification data for each of the 
10 specified locations, wherein the object identification 
data uniquely identifies one of the graphical objects at 
the at least one location; and 
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using the object identification data to identify the 
graphical objects. 

23. The computer-readable medium of claim 22 
wherein at least two of the graphical objects intersect 
at least one location on the two-dimensional display 
device 

24. A method for associating graphical elements in 
a graphical user interface with metadata describing the 
graphical elements, the method comprising: 

associating a unique object identification value 
with each graphical element; 

providing a data structure having an entry for each 
graphical element, each entry being associated with a 
particular object identification value; and 

storing metadata about an associated graphical 
element in a corresponding entry of the data structure. 

25 The method of claim 24 further comprising 
storing the unique object identification value in a frame 
buffer . 

26. The method of claim 24 further comprising: 
determining display information describing how each 

graphical element is to be displayed; 

storing the unique object identification value 
corresponding to the graphical element in a frame buffer 
location associated with the particular display location. 

27. The method of claim 26 further comprising: 
receiving user-input identifying a user-identified 

location in the display; and 

extracting the object identification value 
associated with the user-identified location from the 
frame buffer. 



28. The method of claim 27 further comprising 
retrieving the stored metadata using the extracted object 
identification value. 



29. A computer program product embodied in a 
tangible medium comprising computer program devices 
configured to cause a computer to associate graphical 
elements in a graphical user interface with metadata 

5 describing the graphical elements, the computer program 
product comprising: 

first program devices configured to cause the 
q computer to associate a unique object identification 

Cj value with each graphical element; 

|«* 10 second program devices configured to cause the 

.ri 

"5: computer to implement a data structure having an entry 

Q associated with a particular object identification value; 

7\ and 

M third program devices configured to cause the 

15 computer to store metadata about an associated graphical 

G element in a corresponding entry of the data structure. 

30. The computer program product of claim 29 
further comprising: 

fourth program devices configured to cause the 
computer to determine display information describing how 
5 each graphical element is to be displayed; 

fifth program devices configured to cause the 
computer to select one graphical elements as a front-most 
graphical element at each pixel; and 

sixth program devices configured to cause the 
10 computer to store the unique object identification value 
corresponding to the front-most graphical element in a 
frame buffer location associated with the particular 
display location at which the front-most graphical 
element is displayed. 
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31. The method of claim 30 further comprising: 
seventh program devices configured to cause the 

computer to receive user-input identifying a user- 
identified location in the display; and 

eighth program devices configured to cause the 
computer to extract the object identification value 
associated with the user-identified location from the 
frame buffer. 

32. The method of claim 30 further comprising ninth 
program devices configured to cause the computer to 
retrieve the stored metadata using the extracted object 
identification value. 

33. A system for displaying and interacting with 
graphic objects, the system comprising: 

a display device comprising a plurality of pixels 
arranged in a two-dimensional array, wherein graphical 
objects may be associated with any of the plurality of 
pixels ; 

a frame buffer having a plurality of entries where 
each entry is associated with one of the plurality of 
pixels; and 

object identification information corresponding to 
one of the graphical objects, the object identification 
information being stored in the frame buffer. 

34. A computerized system comprising: 

a display comprising a plurality of pixels where 
each pixel is located at a defined coordinate; 

an application process generating a plurality of 
graphical objects, each graphical object having a unique 
object identification (ID) value; 

a rendering process receiving the plurality of 
graphical objects and determining visible surfaces of the 



plurality of graphical objects with respect to a 
predetermined perspective; 

a pixel map comprising a plurality of entries, each 
entry corresponding to one of the visible surfaces 
determined by the rendering process; 

object identification data associated with at least 
some of the entries in the pixel map; 

a pointer movable to selected coordinates of the 
display, movement of which is controlled by a pointing 
device; and 

an object identification process coupled to read the 
coordinates of a selected pixel pointed to by the pointer 
and extract the object identification information 
associated with the pixel. 

35. A user interface for a three-dimensional 
environment comprising: 

a pointer, movement of which is controlled by a 
pointing device; 

at least one graphical object displayed on at least 
one pixel of a display; 

a frame buffer having memory for the at least one 
pixel, the memory including a first field for holding 
display information and a second field for holding object 
identification information. 

36. A frame buffer comprising: 

a plurality of memory locations, each memory being 
associated with a pixel of a display device; 

display data stored the memory locations, the 
display data indicating how the pixel is to be activated; 
and 

object identification data stored in at least some 
of the memory locations, the object identification data 



indicating a unique software graphical object being 
10 displayed by the pixel. 
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